Nonverbal Communication in Spontaneous Speech Recognition

Verbal communication is the most obvious instrument used to express our thoughts and ideas, considering only this part of speech without regarding its nonverbal part, may lead to overlooking important information of utterance or even misunderstanding it. The development of an automatic system for recognition of facial expressions is a rather difficult task. Such a system must perform automatically, accurately and in real-time by three main steps: face detection and localization, feature extraction and classification. The contributed paper deals with use of automatic system for recognition of facial expressions which have been being created for the Czech dialog system. A prototype of the recognition and understanding system was developed in the Department of Computer Science [8]. The proposed system is fully automatic, user-independent and real-time working. First experiments show that the speech recognition quality is increased by using automatic facial expression recognition system. The work presented in this paper was partly supported by the Grant Agency of Czech Republic under contract number 201/02/1553. Key-Words: Speech understanding, nonverbal communication, face detection and localization, feature extraction and classification.